Coding images

ABSTRACT

An image encoder with a encoder input; a memory device to said encoder input, for storing a image received at said input; a image predictor device for predicting a prediction image based on a first image stored in said memory device; a combiner device for determining residual data relating to a difference between an original image and the prediction image; a first evaluator for determining a first characteristic; an inhibiter connected to said combiner device and said first evaluator device, for transmitting said residual data further if said first characteristic checks against said first predetermined criterion; and a encoder output connected to said inhibiter device. The image encoder has a second evaluator device for determining a second characteristic, and said inhibiter device is arranged for transmitting said residual data further depending on said first characteristic and said second characteristic.

The invention relates to an image encoder according to the introductorypart of claim 1.

In the art of predictive image encoding, such encoders are generallyknown. Motion-compensated video compression generally requires thetransmission of residual information to correct the motion-compensatedprediction. The residual information may be based on a pixelwisedifference between an original frame and a prediction frame.

However, the known encoders are disadvantageous because in case of anerroneous estimate, the amount residual data particularly tends toincrease in detailed areas and hence the amount of outputted data tendsto be large.

It is therefore a goal of the invention to provide an encoder with asmaller amount of outputted data. In order to achieve this goal,according to the invention, an encoder device as described ischaracterized in that said image encoder further comprises: a secondevaluator device for determining at least one second characteristic, andsaid inhibiter device is arranged for transmitting said residual datafurther depending on said first characteristic and said secondcharacteristic.

The amount of outputted data is reduced because the residual data istransmitted depending on the first and second characteristic.Furthermore, if the first characteristic relates to the differencebetween the original image and the predicted image and the secondcharacteristic corresponds to changes of elements in the original imagecompared to the first image, the amount of outputted data is reducedwithout perceived loss of image quality, because if the change ofelements is spatially (and temporally) consistent, small errors in theprediction are not perceived.

The invention further relates to a coding method according to claim 15.As a result of such a method, less data is outputted.

The invention further relates to devices according to claims 16-18,incorporating an image encoder device according to the invention. Also,the invention relates to a coding system according to claim 19, a datacontainer device according to claim 20, a computer program according toclaim 21, a data carrier device according to claim 22 and a signalstream according to claim 23. Such devices, systems and program outputless data.

Specific embodiments of the invention are set forth in the dependentclaims.

Further details, aspects and embodiments of the invention will bedescribed with reference to the attached drawing.

FIG. 1 shows a block diagram of a first example of an embodiment of adevice according to the invention.

FIG. 2 shows a block diagram of a second example of a device accordingto the invention.

FIG. 3 shows a flow-chart of a first example of a method according tothe invention.

FIG. 4 shows a block diagram of an MPEG encoder comprising an example ofan embodiment of a device according to the invention.

FIG. 5 shows an exemplary image.

FIGS. 6,7 illustrate ways to express the threshold value as a functionof the vector inconsistency.

FIG. 8 diagrammatically shows a data transmission device provided with aprediction coder device according to the invention.

FIG. 9 diagrammatically shows a data storage device provided with aprediction coder device according to the invention.

FIG. 10 diagrammatically shows an audio-visual recorder device providedwith a prediction decoder device according to the invention.

FIG. 1 shows a block diagram of an example of an embodiment of anencoder device 10 according to the invention. The encoder device 10 hasan encoder input 11 connected to an memory input 121 of a memory orbuffer device 12. The memory or buffer device 12 is attached with afirst memory output 122 to an predictor input 131 of an image predictordevice 13 and with a second memory output 123 to a second input 153 of acombiner device 15. The predictor device 13 is connected with a firstpredictor output 133 to a first combiner input 151 of the combinerdevice 15. A combiner output 152 of the combiner device 15 is attachedto an inhibiter input 181 of an inhibiter device 18 and a firstcharacteriser input 1611 of a first characteriser device 161. Thepredictor device 13 is further connected via a first predictor output132 to a second characteriser input 1621 of a second characteriserdevice 162. A second predictor output 133 is connected to a secondencoder output 192. The characteriser device 161 is connected with anoutput 1612 to an input 1711 of an evaluator device 17. A characteriseroutput 1622 of the second characteriser device 162 is connected to aninput 1631 of a threshold determiner device 163. An output 1632 of thedevice 163 is connected to a second input 1712 of the AND device 17. Theevaluator device 17 is attached with an output 172 to a control input183 of the inhibiter device 18. An output of the inhibiter device 18 islinked a first encoder output 191.

In use, signals may be received at the encoder input 11 representing twodimensional matrices, e.g. images or frames. Received images are storedin the memory device 12. In this example, an image M_(t), a precedingimage M_(t−1), and a succeeding image M_(t+1) are stored in the memorydevice 12. The words image, frame and matrix are used interchangeably inthis application.

The predictor device 13 may predict a predicted image based on theframes stored in the memory 12. In the example, the predictor device 13may predict a predicted image M_(pred) of the image M_(t) based on thepreceding frame M_(t−1) and the frame image M_(t+1). After prediction,the predicted image M_(pred) is transmitted via the predictor output 132to the combiner input 151, the second evaluator 16 and the secondencoder output 192. The image M_(t) is also transmitted to the combinerdevice 15 from the memory 12.

The combiner device 15 obtains residual data or error data from thecurrent image and the predicted image. The residual data containsinformation about the differences between the predicted image and thecurrent image. The combiner device 15 outputs the error data to theinhibiter 18 and the first characteriser device 161. Based on theresidual data a first characteristic is determined by the first device161. The first characteristic is compared with a first criterion by theevaluator device 17.

The predicted image M_(pred) is also transmitted by the predictor device13 to the second encoder output characteriser. The predictor 13 outputsvector data relating to changes of elements in the current image withrespect to preceding or succeeding images to the second characteriser162. The second evaluator device 162 determines a second characteristicof the predicted image M_(pred). In this example, the secondcharacteristic corresponds to the changes of elements in the current ororiginal image M_(t) compared to the images the prediction imageM_(pred) is determined from, e.g. the preceding image M_(t−1), and thesucceeding image M_(t+1). The second characteristic is transmitted tothe evaluator 17 which checks the second characteristic against a secondcriterion.

The evaluator device 17 compares the signals from the devices 161,162and outputs a binary one signal if both characteristics 161,162 satisfytheir criterion. Otherwise, a binary zero is outputted by device 17.

The signal from the evaluator device 17 controls the inhibiter device18. The inhibiter device 18 prevents the residual data to be transmittedfurther, i.e. the inhibiter discards the residual data, if at thecontrol port 183 a binary one signal is presented. If the signal at thecontrol port 183 is a binary zero, the inhibiter 18 allows the residualdata to be transmitted further to the first encoder output 191.

Thus, the residual data is only transmitted if both the firstcharacteristic and the second characteristic comply with the respectivepredetermined condition. Thereby, the amount of data outputted by theencoder device 10 is reduced. Furthermore, it is found that errors inthe prediction image, due an erroneous estimate in the change of errors,are not perceived by a person viewing the images as long as the localvariation of the change of elements is relatively small. When the localvariation in the change of elements is relatively large, the change ofelements is said to be locally inconsistent. The value of the secondcharacteristic is proportional to this local inconsistency of the changeof elements. Hence, the amount of data transmitted by the encoder isreduced without perceived loss in the image of video quality.

FIG. 2 shows a second example of an image encoder 10′ according to theinvention. Besides the devices of the encoder of FIG. 1, the encoder 10′has a data processing device 14. The device 14 is connected with aninput 141 to the combiner output 152. An output 142 of the device 14 isconnected to the inhibiter input. The device 14 may perform dataprocessing operations, such as quantising the residual data, with aquantiser 144 or transforming the data, for instance from the timedomain to the frequency domain with transformer 143.

FIG. 3 shows a flow-chart of an example of a prediction coding methodaccording to the invention. In a reception step I, images are received.In a storage step II, the received images M_(t±n), M_(t) are stored in abuffer. In a vectorising step III, changes of elements in the currentimage with respect to an other image are determined. In step IV, theconsistency of these changes is determined, which forms a secondcharacteristic. In a prediction step IV, a prediction image M_(pred) ismade of an image M_(t) based on at least one image M_(t±n) stored in thebuffer and the changes of the elements. The images M_(t±n) may bepreceding the image M_(t), succeeding the image M_(t) or a combinationof preceding and succeeding matrices. The predicted image M_(pred) iscombined with the image M_(t) in a combining step V. As a result of thecombining step residual data M_(res) is obtained. In an evaluation stepVI, the residual data is evaluated and checked against a firstpredetermined criterion. Second characteristic is also evaluated andchecked against a second predetermined criterion in the evaluation stepVII. If both criterions are satisfied, the residual data is transmittedfurther in step VIII, else the residual data is discarded in step IX.

The residual data may be determined in any manner appropriate for thespecific implementation. The residual data may for example be thepixelwise difference between the original image M_(t) and the estimatedimage M_(pred), as is for example used in video compressionapplications, and may mathematically be defined as:R(x, y, t)=I _(est)(x, y, t)−I _(orig)(x, y, t),   (1)where R(x,y,t) represents the residual data, I_(est) (x,y,t) theestimated pixel intensity and I_(orig)(x,y, t) the original pixelintensity at matrix position x, y in an image at time instance t.

In the evaluation of the residual data a value may be determined fromthe error or residual data, for example the mean squared error (MSE) themean absolute difference (MAD) or the Sum of Absolute Differences (SAD)may be used. The first evaluator device 161 may for example determinethe MAD, the MSE or the SAD and compare this with a predeterminedthreshold value T. By way of example, the functioning of the evaluatordevice will be described using the MAD, however other measures may beused instead b the evaluator device.

The MAD may be mathematically defined as $\begin{matrix}{{{MAD}( {x,y,t} )} = {\frac{1}{NM}{\sum\limits_{\xi = 1}^{N}\quad{\sum\limits_{\gamma = 1}^{M}\quad{{R( {{x + \xi},{y + \gamma},t} )}}}}}} & (2)\end{matrix}$

In this equation (2), R represents the residual defined in equation (1),and where N and M denote the width and height respectively of thespatial area evaluated in the calculation. The SAD may mathematically bedescribed as the product N·M·MAD, or: $\begin{matrix}{{{SAD}( {x,y,t} )} = {\sum\limits_{\xi = 1}^{N}\quad{\sum\limits_{\gamma = 1}^{M}{{R( {{x + \xi},{y + \gamma},t} )}}}}} & ( 2^{\prime} )\end{matrix}$

As a first predetermined criterion used in the evaluation of theresidual data. the MAD value may be thresholded. As a result of thethresholding a signal representing a binary one value is returned if thelocal MAD has exceeded a perceptible magnitude, mathematicallydescribed: $\begin{matrix}{{S_{MAD}( {x,y,t} )} = \{ \begin{matrix}1 & {{{MAD}( {x,y,t} )} > T_{MAD}} \\0 & {otherwise}\end{matrix} } & (3)\end{matrix}$

As a measure of the movement of elements in the image, the local spatialand/or temporal consistency of motion vectors may be used. The motionvectors may be estimated by the predictor device or the evaluatordevice, as is generally known in the art of image encoding, for examplefrom the Motion Picture Expert Group (MPEG) compression standard. In theexample image of FIG. 5, the areas A-C (e.g. soccer players, ball, etc.)indicate areas where motion occurs and consequently non-zero motionvectors are estimated. The vector inconsistency VI may mathematically beexpressed as $\begin{matrix}{{{{VI}( {x,y,t} )} = \frac{1}{( {{2N} + 1} )( {{2M} + 1} )( {P + 1} )}}\quad{{\sum\limits_{\xi = 1}^{N}\quad{\sum\limits_{\gamma = 1}^{M}{\sum\limits_{\tau}^{P}\quad{{{\overset{arrow}{D}( {x,y,t} )} - {\overset{arrow}{D}( {{x + \xi},{y + \gamma},{t - \tau}} )}}}}}},}} & (4)\end{matrix}$where {right arrow over (D)} represents a 2-dimensional motion vectorthat describes the displacement of elements between two consecutiveframes, N and M respectively represent the horizontal and verticaldimension of the spatial area of evaluation, and where P is the numberof previous vector fields.

As a second predetermined criterion, the vector inconsistency values maybe thresholded. A signal representing a binary one is then returned ifthe vector inconsistency VI has exceeded a perceptible magnitude.$\begin{matrix}{{S_{VI}( {x,y,t} )} = \{ \begin{matrix}1 & {{{VI}( {x,y,t} )} > T_{VI}} \\0 & {otherwise}\end{matrix} } & (5)\end{matrix}$

Errors (i.e. S_(MAD)=1) are only perceived by a viewer if the motionvectors are locally inconsistent. Thus, as a inhibiting criterion it maybe demanded that both the MAD and the VI must be above the respectivethreshold, i.e. S_(MAD) and S_(VI) have a value of one. In amathematical way this condition may be described as:S _(perceived)(x,y,t)=S _(MAD)(x,y,t){circumflex over ( )}S_(VI)(x,y,t),   (6)where ‘{circumflex over ( )}’ denotes the Boolean ‘AND’ operation.

In the resulting selection only in areas of a strong motion-vectorinconsistency (e.g. soccer players and ball, see also FIG. 5, areas A-C)the MAD errors are identified as being perceivable. The MAD errors inthe audience in the upper part of the frame of FIG. 5 are caused by anerroneous but consistent motion compensation in which the smalldeviation from the true motion are not perceived. Possible alternativecriteria can be obtained by any other linear or non-linear combinationof the MAD and the vector inconsistency values. For example, the localspeed may be included as an additional parameter.

If as a measure of the vector inconsistency the definition of equation(4) is used, a short edge in the vector field may give rise to a lowVI-value. Consequently, a spatially small disturbance (or edge) in thevector field may stay undetected, whereas the errors due to spatiallysmall inconsistencies are generally easily perceived.

An alternative way to describe the temporal consistency of the motionvectors is not to determine the mean absolute vector difference butmaximum absolute vector difference instead, $\begin{matrix}{{{VI}( {x,y,t} )} = {\max\limits_{{- N} \leq \xi \leq {N\bigwedge{- M}} \leq \gamma \leq {M\bigwedge 0} \leq r \leq P}\quad{{{\overset{arrow}{D}( {x,y,t} )} - {\overset{arrow}{D}( {{x + \xi},{y + \gamma},{t - \tau}} )}}}}} & (7)\end{matrix}$

The vector consistency calculated with equation (7) treats all thevector differences within the ‘kernel’ range equally important,disregarding the number of vector elements that contribute to thedifference. The vector-inconsistency calculated according to equation(7) tends to contain broad regions of high VI-values around adisturbance (or edge). The spatial (temporal) dimensions of the area ofthe high VI-value is determined by the kernel-size parameters N, M, andP.

The first criterion may be related to the second characteristic or thesecond criterion. For example, the threshold for the MAD may be relatedto the vector inconsistency. For example, if the vector inconsistency isdetermined using expression (7) instead of equation (4), the MAD valuesmay be thresholded using the vector inconsistency values,T _(MAD)(x,y,t)=α(VI _(max) −VI(x,y,t))   (8)with α being a positive multiplication factor, and VI_(max)=2|{rightarrow over (D)}|_(max) being the maximum of possible VI-values. Thethreshold T_(max) is inversely proportional to the vector inconsistencyVI. FIG. 6 shows the threshold as a function of the vector inconsistencyas described by equation (8). The relation between the threshold T_(MAD)and the vector inconsistency VI does not need to linear. In general, thefunction T_(MAD)(VI) may be any non-ascending function, and may beimplemented as a analytical function or a look-up table.

The behavior of a fixed threshold, i.e. the use of equations (3) and (5)may be achieved using the function depicted in FIG. 7. T_(MAD)(fixed) isthe value T_(MAD) in equation (3), T_(VI)(fixed) is the value T_(VI) inexpression (₅). If VI>T_(VI)(fixed) and MAD>T_(MAD)(fixed), the residualdata is omitted.

Both the residual data and the movement of elements in the image may bedetermined and evaluated on a block basis, as is for example known fromMPEG compliant image encoding. The invention may be applied in a methodor device according to an existing video compression standard. Existingvideo compression standards such as MPEG-2are commonly based onmotion-compensated prediction to exploit temporal correlation betweenconsecutive images or frames, see e.g. [1,2].

In MPEG, decoded frames are created blockwise from motion-compensateddata blocks obtained from previously transmitted frames. Themotion-compensated predictions may be based either on the previous framein viewing order, or both on the previous and the next frame in viewingorder. The unidirectional predictions and the bi-directional predictionsare referred to as P-frames and B-frames respectively. The use ofB-frames requires temporal rearrangement (shuffling) of the frames suchthat the transmission order will not be equal anymore to the viewingorder. The residual data may outputted by the encoder to correct forerrors in the motion-compensated prediction, both for the P- andB-frames.

FIG. 4 shows an example of an image encoder device 100 compliant withthe Motion Pictures Expert Group 2 (MPEG-2) standard. The image encoderdevice is referred to as the MPEG-encoder device 100 from this point on.In the shown encoder device 100 B-frames are predicted. However, I- orP-frames may be used instead.

The MPEG encoder device 100 has an encoder input 11 for the reception ofvideo images. Connected to the encoder input 11 is a memory input 121 ofa memory device 12. The memory device 12 has a first memory output 122connected to a first predictor input 131 of a predictor device 13. Afirst output 132 of the predictor device 13 is connected to a firstinput 151 of a first combiner device 15. A second memory output 123 isconnected to a second combiner input 153 of a first combiner device 15.A combiner output 152 is connected to a switch input 181 of a switch 18.A switch output 182 is connected to a discrete cosine transformer device(DCT) 20 via a DCT input 201. A DCT output 202 of the DCT 20 isconnected to an quantiser input 211 of a quantiser device 21. Thequantiser device 21 is attached with a quantiser output 212 to an input231 of a skip device 23. The skip device 23 is connected to a variablelength coder device (VLC) 24 via a skip output 232 and a VLC input 241.An output of the VLC 24 is attached to an encoder output 19.

The quantiser device 21 is also attached with the quantiser output 212to an inverse quantiser (IQ) input 221 of an inverse quantiser device(IQ) 22. The IQ 22 is connected with an IQ output 222 to an input 251ofan inverse discrete cosine transformer device (IDCT) 25. The IDCT 25 isattached to a first combiner input 151′ of a second combiner device 15′via an IDCT output 252. The second combiner device 15′ is also connectedwith a second combiner input 153′ to a predictor output 132 of thepredictor device 13. An output 152′ of the second combiner device 15 isconnected to a second input 133 of the predictor device 13. A secondpredictor output 139 of the predictor device 13 is connected to a firstevaluator input 1601 of an evaluator device 16 which is also connectedto the combiner output 152 of the first combiner device via a secondevaluator input 1603. An evaluator output 1602 is connected to a switchcontrol input 183 of the switch 18 and a skip control input 233 of theskip device 23.

In use, signals representing images may be received at the MPEG-2encoder input 11. The received images are stored in the memory device 12and transmitted to both the predictor device 13 and the first combinerdevice 15. The predictor device may predict B-frames, so in the memory12, the order of the received images may be rearranged to allow theprediction.

The predictor device 13 predicts an image, based on preceding and/orsucceeding images. The first combiner device 15 combines the predictedimage with an original image stored in the memory 12. This combinationresults in residual data containing information about differencesbetween the predicted image and the original image. The residual data istransmitted by the first combiner device 15 to the evaluator device 16and the switch device 18.

In a connected state the switch input and the switch output arecommunicatively connected to each other. In a disconnected state, theswitch input and the switch output are communicatively disconnected. Thestate of the switch is controlled by a signal presented at the switchcontrol input 183. In the example of FIG. 3, the evaluator devicecontrols the state of the switch 18. In the connected state, the switchdevice transmits the residual data to the DCT device 20. In should benoted that the switch 18 may be omitted, in the shown example, theswitch 18 is implemented in the encoder to avoid useless processing ofresidual data may be discarded by the skip device 23.

The DCT 20 may convert the residual data signals from the spatial domaininto the frequency domain using discrete cosine transforms (DCTs).Frequency domain transform coefficients resulting from the conversioninto the frequency domain are provided to the quantiser device 21.

The quantiser device 21 quantises the transform coefficients to reducethe number of bits used to represent the transform coefficients andtransmits the resulting quantised data to the skip device 23.

The skip device 23 decides may insert a skip macro-block escape code ora coded-block pattern escape code as is defined in the MPEG-2 standard,depending on the signal presented at the skip control input 233.

The variable-length coder 24 subjects the quantised transformcoefficients from the quantiser 21 (with any inserted skip code) tovariable-length coding, such as Huffmann coding and run-length coding.The resulting coded transform coefficients, along with motion vectorsfrom the predictor 13, are then fed as a bit stream, via the outputbuffer 19, to a digital transmission medium, such as a Digital VersatileDisk, a computer hard disk or a (wireless) data transmission connection.

The predictor device comprises 13 two memories 134,135 (MEM fw/bw andMEM bw/fw) which are connected to the second predictor input 133 with arespective memory input 1341,1351. The memories 134,135 contain aprevious I- or P-frame and a next P-frame. The frames in the memoriesare transmitted to a motion estimator device (ME) 136 and amotion-compensated predictor device (MC) 138, which are connected withtheir inputs 1361,1381 to outputs 1342,1352 of the memories. Of course,the memories 134,135 may likewise be implemented as a single memorydevice stored with data representing both frames.

The ME 136 may estimate motion-vector fields and transmit the estimatedvectors to a vector memory (MEM MV) 137 connected with an input 1371 toa ME output 1362. The motion estimation (ME) may be based on the framesstored in the memories 134,135 or on the frames in the shuffling memory12. The vectors stored in the MEM MV 137 are supplied to a motioncompensated predictor 138 and used in a motion compensated prediction.The result of the motion compensated prediction is transmitted to thefirst predictor output 132.

The vectors stored in the MEM MV 137 are also transmitted to the secondoutput 139 of the predictor device 13. The vectors are received via thefirst estimator input 1601 by a vector inconsistency estimator 162 inthe evaluator device 16. The vector inconsistency estimator performs anoperation described by equation (4) or equation (7) for the vectors fromthe vector memory 137.

A second evaluator input 1602 connects a SAD device 161 to the combineroutput 152 of the first combiner device 15. The SAD device 161determines the SAD as described by equation 2′ from the residual data atthe output of the first combiner device 15. Both the SAD and the vectorinconsistency are thresholded by the respective devices. The result ofthe thresholding is fed into an AND device 163, which performs an ANDoperation as is described above. The output signal of the AND device 163is fed to the control inputs of the switch device 18 and the skip device23.

Thus, the combined values of the SAD and vector inconsistency result ina binary decision (error criterion) per macro-block whether to transmitthe residual data or not and whether to insert a macro-block skip codeor not. The magnitude and direction of the estimated motion vectors maylocally deviate from the true motion. In the case of a small localdeviation of the motion vectors, the local value of R depends on thelocal level of high-frequent detail. In the case of high level localdetail, the high numerical value of R suggests that it is necessary tolocally ‘repair’ the erroneous estimate. However, in case a motionvector deviation extends over a large area, and in case the directionand magnitude of the deviation is consistent within that area, smalldeviations are not perceived.

The binary decision is used to avoid further calculation of the DCT andresults in the generation of the skip macro-blocks escape code (skipMBs) and coded-block pattern escape codes to skip empty DCT-blockswithin one macro-block. The use of the skip macro-blocks and coded-blockpattern escape code results in an efficient description of residualframe data With the new criterion, the encoder still produces an MPEG-2bitstream which can be decoded by every MPEG-2-compliant decoder. Thebinary decision is used to avoid further calculation of the DCI andresults in the generation of the skip macro-blocks escape code (skipMBs) and coded-block pattern escape codes to skip empty DCT-blockswithin one macro-block.

Both the MPEG-2 bitstream and the proprietary residual stream aremultiplexed to form one MPEG-compliant stream. The MPEG-2 standardsupplies the possibility to use a so called private data channel forproprietary data. In case no residual data is transmitted only theBoolean map S_(perceived)(x,y,t) is transmitted. The use of the skipmacro-blocks and coded-block pattern escape codes enables us to avoid aseparate transmission of the Boolean map S_(perceived)(x,y,t). Thepattern of the escape codes within each frame implicitly holds theinformation to regenerate the Boolean S_(perceived)(x,y,t).

The motion estimator may be of any type suitable for the specificimplementation. In video coding, most methods for motion estimation arebased on full-search block matching (FSBM) schemes or efficientderivations thereof. The motion estimation may also be based on aestimation method known from frame-rate conversion method. In suchmethods frames are temporally interpolated using previous and futureframes, similar to video compression. However, since no residual data isavailable, the correct motion vector field for frame-rate conversionalmost always represents the true motion of the objects within the imageplane. The method of three-dimensional recursive search (3DRS) is theprobably the most efficient implementation of true-motion estimationwhich is suitable for consumer applications [3,4,5,6,7]. Themotion-vectors estimated using 3DRS tend to be equal to the true motion,and the motion-vector field inhibits a high degree of spatial andtemporal consistency. Thus, the vector inconsistency is low, whichresults in a high threshold to the SAD values. Since the SAD-values arenot thresholded very often, the amount of residual data transmitted isreduced compared to the non-true motion estimations.

The invention may be applied in various devices, for example a datatransmission device 40 as shown in FIG. 8, like a radio transmitter or acomputer network router that includes input signal receiver means 41 andtransmitter means 42 for transmitting a coded signal, such as an antennaor an optical fibre. The data transmission device 40 is provided withthe image encoder device 10 according to an embodiment of the inventionthat is connected to the input signal receiver means 41 and thetransmitter means 44. Such a device is able to transmit a large amountof data using a small bandwidth since the data is compressed by theencoding process without perceived loss of image quality.

It is equally possible to apply the image encoder device 10 in a datastorage device 30 as in FIG. 9, like an optical disk writer, for storingimages on a data container device 31, like a SACD, a DVD, a compact discor a computer hard-drive. Such a device 30 may include holder means 32for the data container device 31, writer means 33 for writing data tothe data container device 31, input signal receiver means 34, forexample a microphone and a prediction coder device 1 according to theinvention that is connected to the input signal receiver means 34 andthe writer means 33, as is shown in FIG. 10. This data storage device 30is able to store more data, i.e. images or video on a data containerdevice 31, without perceived loss of image or video quality.

Similarly, an audio-visual recorder device 60, as shown in FIG. 10,comprising audiovisual input means 61, like a camera or a televisioncable, and data output means 62 may be provided with the image encoderdevice 10 thereby allowing to record more images or video data whileusing the same amount of data storage space.

Furthermore, the invention can be applied to data being stored to a datacontainer device like a floppy disk a Digital Versatile Disc or a SuperAudio CD, or a master or stamper for manufacturing DVDs or SACDs.

The invention may also be implemented in a computer program for runningon a computer system, at least including code portions for performingsteps of a method according to the invention when run on a computersystem or enabling a general propose computer system to performfunctions of a computer system according to the invention. Such acomputer program may be provided on a data carrier, such as a CD-rom ordiskette, stored with data loadable in a memory of a computer system,the data representing the computer program. A data carrier may furtherbe a data connection, such as a telephone cable or a wireless connectiontransmitting signals representing a computer program according to theinvention.

In the foregoing specification, the invention has been described withreference to specific examples of embodiments of the invention. Thespecifications and drawings are, accordingly, to be regarded in anillustrative rather than in a restrictive sense. It will, however, beevident that various modifications and changes may be made thereuntowithout departing from the broader spirit and scope of the invention asset forth in the appended claims.

For example, the invention is not limited to implementation in thedisclosed 5 examples of devices, but can likewise be applied in otherdevices. In particular, the invention is not limited to physical devicesbut can also be applied in logical devices of a more abstract kind or ina computer program which enables a computer to perform functions of adevice according to the invention when run on the computer.

Furthermore, the devices may be physically distributed over a number ofapparatuses, while logically regarded as a single device. Also, deviceslogically regarded as separate devices may be integrated in a singlephysical device having the functionality of the separate devices.

References

-   [1] D. L. Gall, “MPEG: A video compression standard for multimedia    applications,” Communications of the ACM, vol. 34, no. 4, pp. 46-58,    1991.-   [2] J. L. Mitchell, W. B. Pennebaker, C. E. Fogg, and D. J. LeGall,    MPEG Video Compression Standard. Digital Multimedia Standards    Series, New York, N.Y.: Chapman & Hall, 1997.-   [3] G. de Haan and H. Huijgen, “Method of estimating motion in a    picture signal.” U.S. Pat. No. 5,072,293, December 1991.-   [4] G. de Haan and H. Huijgen, “Motion vector processing device.”    U.S. Pat. No. 5,148,269, September 1992.-   [5] G. de Haan and H. Huijgen, “Apparatus for motion vector    estimation with asymmetric update region.” U.S. Pat. No. 5,212,548,    May 1993.-   [6] G. de Haan, P. W. A. C. Biezen, H. Huijgen, and O. A. Ojo,    “True-motion estimation with 3-D recursive search block matching,”    IEEE transactions on Circuits and Systems for Video Technology, vol.    3, pp. 368-379, October 1993.-   [7] G. de Haan and P. W. A. C. Biezen, “Sub-pixel motion estimation    with 3-D recursive search blockmatching,” Signal Processing: Image    Communication, vol. 6, pp. 229-239, 1994.-   [8] U.S. Pat. No. 5,057,921.

1. An image encoder (10;100), at least comprising: at least one encoderinput (11); at least one memory device (12,121-123) connected to saidencoder input, for storing at least one image received at said input; atleast one image predictor device (13,131-139) for predicting aprediction image based on at least one first image stored in said memorydevice; a combiner device (15,151-153) for determining residual datarelating to a difference between an original image and the predictionimage; a first evaluator device (161,1611-1613) for determining at leastone first characteristic; an inhibiter device (18,181-183; 23,231-233)connected to said combiner device and said first evaluator device, fortransmitting said residual data further if said first characteristicchecks against said first predetermined criterion; and at least oneencoder output (19) connected to said inhibiter device, characterized inthat: said image encoder (10;100) further comprises: a second evaluatordevice (162,1621-1622) for determining at least one secondcharacteristic, and said inhibiter device (18,181-183; 23,231-233) isarranged for transmitting said residual data further depending on saidfirst characteristic and said second characteristic.
 2. An image encoder(10;100) as claimed in claim 1, wherein said first evaluator device(161,1611-1613) is arranged for determining at least one firstcharacteristic relating to said difference and a second evaluator device(162,1621-1622) for determining at least one second characteristiccorresponding to changes of elements in said original image compared tosaid at least one first image.
 3. An image encoder (10;100) as claimedin claim 1, wherein said first evaluator device (161,1611-1613) isarranged for checking said first characteristic against a firstcriterion, said second evaluator device (162,1621-1622) is arranged forchecking said second characteristic against a second criterion and saidinhibiter device is arranged for transmitting said residual data furtherif said first characteristic checks against said first criterion andsaid second characteristic checks against said second criterion.
 4. Animage encoder (10;100) as claimed in claim 3, wherein said firstevaluator device (161) comprises: an average device for determining anaverage prediction error a first comparator device for comparing saidaverage prediction error with an error threshold value.
 5. An imageencoder as claimed in claim 3, wherein said image predictor devicecomprises: at least one motion vector estimator device (136, 1361-1363),for predicting motion vectors relating to position changes of elementsin said image; and said second evaluator device (162)comprises: a motionvector inconsistency estimator and a second comparator device forcomparing said motion vector inconsistency value with a predeterminedinconsistency threshold value.
 6. An image encoder (10;100) as claimedin claim 5, wherein said motion vector inconsistency estimator device isarranged for performing an operation represented by the mathematicalalgorithm:${{VI}( {x,y,t} )} = \frac{1}{( {{2N} + 1} )( {{2M} + 1} )( {P + 1} )}$$\quad{\sum\limits_{\xi = 1}^{N}\quad{\sum\limits_{\gamma = 1}^{M}\quad{\sum\limits_{\tau}^{P}\quad{{{\overset{arrow}{D}( {x,y,t} )} - {\overset{arrow}{D}( {{x + \xi},{y + \gamma},{t - \tau}} )}}}}}}$wherein VI represents said vector inconsistency, {right arrow over (D)}represents a motion vector, N represents a horizontal dimension of aarea of evaluation of said prediction image, M represents a verticaldimension of said spatial area, P represents is a number of previousvector fields.
 7. An image encoder (10;100) as claimed in claim 5,wherein said motion vector inconsistency estimator device is arrangedfor performing an operation represented by the mathematical algorithm:${{VI}( {x,y,t} )} = {\max\limits_{{- N} \leq \xi \leq {N\bigwedge{- M}} \leq \gamma \leq {M\bigwedge 0} \leq r \leq P}\quad{{{\overset{arrow}{D}( {x,y,t} )} - {\overset{arrow}{D}( {{x + \xi},{y + \gamma},{t - \tau}} )}}}}$wherein VI represents said vector inconsistency, {right arrow over (D)}represents a motion vector, N represents a horizontal dimension of aarea of evaluation of said prediction image, M represents a verticaldimension of said spatial area, P represents is a number of previousvector fields.
 8. An image encoder (10;100) as claimed in claim 5,wherein said second comparator is arranged for performing an operationrepresented by the mathematical algorithm:${S_{VI}( {x,y,t} )} = \{ {\begin{matrix}1 & {{{VI}( {x,y,t} )} > T_{VI}} \\0 & {otherwise}\end{matrix}.} $ where S_(VI) represents a binary valueindicating the outcome of said checking, and T_(VI) represents saidpredetermined threshold value.
 9. An image encoder (10;100) as claimedin claim 1, wherein said prediction image is an interpolated image,predicted from at least one preceding image preceding said originalimage and at least one succeeding image succeeding said original image.10. An image encoder (10;100) as claimed in claim 9, wherein saidinterpolated image is an MPEG B-frame image.
 11. An image encoder(10;100) as claimed in claim 5, wherein said motion estimator device(136,1361-1363) is a true motion estimator device.
 12. An image encoder(10;100) as claimed in claim 3, wherein the first criterion is relatedto the second characteristic
 13. An image encoder as claimed in claim12, wherein the vector inconsistency is used to compute said firstthreshold according to the mathematical algorithm:T_(MAD(x,y,t))=α(VI_(max)−VI(x,y,t)) with a positive multiplicationfactor, and VI_(max)=2|{right arrow over (D)}|_(max) being the maximumof possible VI-values.
 14. An MPEG compliant image encoder (100),comprising at least one image encoder as claimed in claim
 1. 15. Acoding method, comprising: receiving (I) at least one first image and anoriginal image; predicting (IV) a prediction image based on said atleast one first image; determining (V) residual data relating to adifference between the original image and the prediction image; andtransmitting(VIII) said residual data further if at least onepredetermined criterion is satisfied, characterized in that: said atleast one criterion comprises: determining (V) at least one firstcharacteristic; and determining (III, VI) at least one secondcharacteristic.
 16. A data transmission device (40) comprising inputsignal receiver means (41), transmitter means (42) for transmitting acoded signal and an image encoder device (10) as claimed in claim 1connected to the input signal receiver means and the transmitter means.17. A data storage device (30) for storing data on a data containerdevice (31), comprising holder means (32)for said data container device,writer means (33) for writing data to the data container device, inputsignal receiver means (34) and an image encoder device (10) as claimedin claim 1 connected to the input signal receiver means and the writermeans.
 18. An audiovisual recorder device (60), comprising audiovisualinput (61) means, data output means (62) and an image encoder device(10) as claimed in claim
 1. 19. A coding system, comprising: an encoderdevice a decoder device communicatively connected to said encoderdevice, characterized in that said encoder device comprises at least oneinverse an image encoder device as claimed in claim
 1. 20. A datacontainer device containing data representing images coded with a imageencoder device as claimed in claim
 1. 21. A computer program includingcode portions for performing steps of a method as claimed in claim 15.22. A data carrier device including data representing a computer programas claimed in claim
 21. 23. A signal stream representing encoded images,said stream including data representing at least one predicted image andsaid stream containing residual data relating to a difference betweensaid predicted image and an original image depending on a firstcharacteristic and a second characteristic of if at least one firstvalue corresponding to said difference checks against a first criterionand at least one second value corresponding to a predicted change ofelements in said predicted image checks against a second criterion.